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Abstract 

In this note, we establish superiority of the so-called copositive 
bound over a bound suggested by Nesterov for the problem to 
minimize a quadratic form over the r^-balh We illustrate the 
improvement by simulation results using Jos Sturm’s SeDuMi. 
The copositive bound has the additional advantage that it can be 
easily extended to the inhomogeneous case of quadratic objectives 
including a linear term. We also indicate some improvements of 
the eigenvalue bound for the quadratic optimization over the 
ball with 1 < p < 2, at least for p close to one. 
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1 Introduction 


As usual we denote for p > 0 by ||j/||p = [X]r=i ^^e p-norm of a 

vector y in n-dimensional Euclidean space The £^-ball is then Bp = 

I?/ G M"" : lli/llp — ^he Handbook of Semidefinite Programming j7j, 

Nesterov deals with the problem of minimizing a quadratic form over the 
£^-ball -Bi, among many others (see p. 387 in [Zj), and specifies a bound 
obtained by SDP relaxation for this problem. However, the feasible set Bi is 
a polytope with not too many, and known, vertices, namely ie*, 1 < i < n, 
where e* are vectors in with one as i-th coordinate and zeroes elsewhere. 
This elementary observation allows for transformation of the problem into a 
moderately sized Standard Quadratic Optimization Problem (StQP), which 
consists of minimizing a quadratic form over the standard simplex 

= {xE M+ : e^x = l} , 

where e = [e"*", E and e = E M" are all-ones vectors in and 
R”, respectively (R^ denotes the set of all vectors in R”^ with no negative 
coordinates). This way, we may apply any valid bound for StQPs also to 
the quadratic problem over Bi. Here, we focus on the so-called copositive 
(relaxation) bound introduced in [B] and recently investigated in more detail 
in p. 

The paper is organized as follows: First, we present the reformulation of 
the problem as a StQP, then investigate the copositive bound for this spe¬ 
cial problem class and establish superiority compared to the bound proposed 
by Nesterov. We also address the special case of sign-constrained data ma¬ 
trices where it can be shown that the size of the StQP can be kept to the 
original size, so that - unlike the general case - doubling the dimension can 
be avoided. Empirical quality assessment is provided by a small simulation 
study. Finally, we use the established copositive bound to obtain a lower 
bound when minimizing a quadratic form over the £^-ball Bp for 1 < p < 2. 

2 Reformulation as an StQP 

Given a symmetric, possibly indefinite n x n matrix C and a vector c G R"", 
consider the quadratic minimization problem over the £^-ball 

a* = min {y^Cy + 2c^y : y E Bi] . (1) 

Since the feasible set Bi of the above problem is the convex hull of the vectors 
icj introduced above, we can write 

a* = min \^x^{V^CV + ec^V -|- ce)x : x E A^"'} , (2) 
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where V = [/„, —/„] contains all vertices of the £^-ball as column vectors. 
Hence, introducing the symmetric (2n) x (2n) matrix 


c 

-C ■ 

+ 

ec~^ + ce~^ ce~^ — ec~^ 

-c 

c 

ec~^ — ce~^ —ec~^ — ce~^ 


we arrive at the StQP 

a* = min {x~^Qc,cX : x E , 

and any StQP bound for Qc,c is a valid bound for a*. Here, we will focus on 
the so-called copositive relaxation bound. For the readers’ convenience, we 
provide some background in the following section. 

3 Copositive relaxation bounds for StQPs 

As is well known j3], we can reformulate every StQP in m-dimensional space 
of the form 

Oq = min [x~'^Qx : x E A™'} (3) 

into a copositive program: 

max {A : Q — XE E C} = aq , (4) 

where E is the all-ones m x m matrix and 

C = [A E S : x~^Ax > 0 for all x E (5) 

denotes the convex cone of all copositive matrices. Here S are the symmetric 
m X m matrices. 

Let now V E S denote the cone of all positive semidefinite matrices and 
M E S the cone of all nonnegative symmetric matrices. Then, a (zero-order) 
approximation jB] of the copositive cone is given hy Kq = V + M ^ C with 
C = /Co only if n < 4 jH] . Replacing C with /Co yields the copositive relaxation 
bound: 

= max {A : Q — XE E JCq = V + M} < aq . (6) 

Passing to the dual problems of dH) and we obtain alternative for¬ 
mulations for aq and respectively. To this end, we need some more 
notation. Let A, B be two symmetric m x m matrices and recall that the 
trace of a matrix is the sum of its diagonal elements. Then define the inner 
product 

A* B = trace (HR) = atjbij . 

ij 
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Now strong duality arguments yield equality of m below with (jH), and 
with 0 below, respectively. For details see [H]. 

mm{Q»X : E»X = 1, X eC*} = Oq, (7) 

with C* the completely positive cone, the dual cone of C given by 

C* = G 5 : i/i e M+ , i e {1,..., fc} , some /c| , (8) 

and 

mm{Q • X : E • X = 1, X e V nX} = , (9) 

&s Xq = V + X has the dual cone Xq = V CiX D C*. 

A direct argument why the solution of (jHI) never can exceed Og employs 
the fact that for any x G A™, the rank-one matrix X = xx'^ satisfies X E Xq 
as well as E • X = xY = 1, along with Q • X = x~^Qx. 


4 Copositive relaxation bounds 
for the ^^-constrained problem 


Now let us specialize to Q = Qc,c, where m = 2n. We focus on the dual 
formulation and partition any X G P fl A/” in a natural way: 


Straightforward calculations then yield 

Qc,c»X = C»{U-Y-Y^ + V) + {eX+ce^)»{U-V)-{eX-ce'^)»{Y-Y^). 


In order to compare the copositive bound with the bound suggested by Nes¬ 
terov we will restrict ourselves to the homogeneous case c = o, and abbreviate 
Qc = Qc,o ■ Then the copositive bound is given by 


a 


cop 

Qc 


= imn{C •{U -Y - Y^ + V) ■. X = 


U Y 
Y^ V 


G vnX, E»x = 1 

( 10 ) 

On p. 387 of [7], Nesterov proposes an alternative SDP relaxation bound 
as follows (in maximization rather than minimization form): 


a 


Nest 

C 


= min {C • W : Diag {u) YW Y O for some u G A"'} (11) 
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(actually he requires e^u < 1 rather than e^u = 1 as included in the dehni- 
tion of A"', but obviously, scaling u such that e^u = 1 holds does not change 
(^^-feasibility). Here and in the sequel, we abbreviate hj A ^ B the fact 
that A — B is positive-semidefinite. 

Our aim is now to show that always < a* holds, where the 

last inequality follows from validity of the general StQP bound. Note that in 
the (easy convex) case of positive semidefinite O, all three values are equal 
to 0. In order to prove the first inequality in the general case, we start with 
an auxiliary result. 


Lemma 1 Let 


X 


U Y 
V 


be a symmetric positive-semidefinite matrix where all sub-matrices are square 
and of the same size, and U, V and Y have no negative entries. Define 


X_ 


U -Y 
-Y^ V 


and let D = Diag {Xe) be the diagonal matrix containing the row-sums of X 
on the diagonal. Then 

(a) X_ is positive-semidefinite; and 

(b) D — X_ is positive-semidefinite. 


Proof. First we show that X G P if and only if X_ G V. Because of the 
special structure of X and X_, we have that for any two vectors a and b 


[a^ 6^ ] X_ 


a 

b 


= a^Ua-2a^Yb + b'^Vb= [a'^ -b'^ ] X 


a 

-b 


( 12 ) 

Hence assertion (a) is established. Furthermore, since D is a diagonal matrix, 
it has only zero off-diagonal blocks. Therefore, by the same argument as 
before, D — X G P if and only HD — X_ G P, so it remains to show that 
D — X is positive-semidefinite. 

To this end we interpret X as the adjacency matrix of an edge-weighted 
graph, and D — X as its corresponding Laplacian matrix, which is well known 
to be positive-semidefinite, if there are no negative weights. For the conve¬ 
nience of the reader we give a short proof, for more details see e.g. [2]. The 
Laplacian D — X can be rewritten as BGB^, where B is the oriented inci¬ 
dence matrix and G is a diagonal matrix containing the weights of the graph. 
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Indeed, indexing the columns of the m x matrix B by ordered pairs {k, 1) 
with 1 < fc < / < m, we have 



and correspondingly 


G 


{k,l),{r,s) 


0 


(fc,o 


if i = k 
if i = I 
otherwise 


if {k,l) = (r,s) 
otherwise. 


It is then straightforward to show that BGB^ = D — X. Now, since X has 
no negative entries, we can take the square root G^ of the diagonal matrix 
G and thus obtain D — X = {BG^){BG^)~^ ^ O. □ 


Theorem 4.1 We have < o-q^ for all symmetric n x n matrices G, 
with strict inequality for some instances. 

Proof. We show that for any ms -feasible X, the matrix W = U —Y — Y^ + 
V together with the vector u = {U + Y + Y~^ + V)e satisfies the constraints of 
(HH). Then the assertion follows immediately by definition of the problems 
defining the two bounds. 

First, Lemma ^a) ensures positive semidefiniteness of W, since y~^Wy = 
[y^,y~^]X_[y'^,y~^]^ > 0. Next, u G by the nonnegativity assumption 
on U, Y, and V, and e^u = e^Xe = E • X = 1, whence u G A"" results. 
Further, D = Diag (Xe) satisfies 


2 /^(Diag u)y = J2[{U + Y)e],y^ + + V)e],y^ = [y^, y^]D 


y 

y 


whence |/'''(Diag u — W)y = [y'^,y'^]{D — X_)[y'^,y'^]^ > 0 for all ?/ G M” 
results, due to Lemma m- Therefore we conclude Diag u Y W Y O, and 
(IF, u) is ([TTJ-feasible. Finally, to establish strict domination of the copositive 
bound, it suffices to look at the 3x3 instance 


G = 


—1 a 

a —1 

-1 -1 


-1 

-1 

-1 


(13) 


For a = 1 we get = —4/3 whereas = — 1 = a* is even exact. □ 
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Thus, by using a StQP reformulation of the original problem we obtain a 
better SDP bound. In case one does not want to solve the SDP relaxations 
to optimality and still needs a valid bound, the dual formulations could be 
preferred, which give a valid bound for any feasible objective value. For 
completeness, we specify them for both bounds: 

= max |A G M : A < {Qc)ij — Sij for all i, j , and some ^ ^ 0} 

which is the originally primal form and the dual formulation of the 
Nesterov bound which with small modifications appears already in [7]: 

^Nest _ {A G M : diag (S') < —Ae, S ^ —C for some S ^ 0} . 

A direct comparison of these two formulations does not seem immediately 
evident. 

The price we have to pay for more efficiency of is the double dimen¬ 
sion. One may wonder what happens if the {2n) x (2n) matrix Qc is replaced 
by C itself, thus arriving at an SDP of the same size as that used in 


Theorem 4.2 For any symmetric C, we have 


a 


cop 


< a 


cop 


"Qc - ’ 

but in general, the latter also exceeds the true value a* 


Proof. Decompose any (UHl) -feasible X again as 




U Y 

yT Y 


If, in particular, Y = V = O, we arrive at O • (f/ — 2Y + V) = C • U, and 
the conditions X e V H J\f and E • X = 1 boil down to U G P fl A/” (by 
abuse of notation, we ignore differences of matrix size in the cones) as well 
as ee"'" • U = 1, so that 

< min |0 • U : ee"'" •17 = 1, U (1 A/”} = a™*’, 

which proves the assertion. For the matrix C of m with a = 2 we have 
-1.5 = a* < = -1. □ 
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5 Sign-constrained data matrices 

Here we focus on the particular case where C has no positive entries, i.e. 
where —C G M. In this case the minimum of the quadratic form over the 
£^-ball is attained at a point of the standard simplex: 

Proposition 5.1 For any symmetric C with no positive entries we have 

a* = min \^x~^Cx : a; G -Bi} = min \^x~^Cx : x G A} . 

Proof. First note that except for the trivial case C = O a. nonnegative ma¬ 
trix cannot be positive semidehnite. Therefore it is guaranteed that a* is at¬ 
tained at a point x on the boundary of Bi, whence we get 1^*1 = 1- Then 
X* := [|xi|,..., Ixnl]"’" G A"' and from —C G A/'it follows {x*yCx* < {xYCx, 
and thus the proposition. □ 


This relation has an exact counterpart for the SDP-relaxation: we can indeed 
use the cheaper copositive bound considered at the end of the previous 
section, because the latter coincides with 

Theorem 5.1 For any symmetric C with no positive entries, we have 


cop cop ^ * 

and again, in general, a™’’ > Og®®* for some instances C G —Af. 

Proof. In view of Theorem ia we only have to show > Og*’. Now, if 


A 


U Y 
V 


ePnJY, 


and also -C e J\f, we get C • (U - Y - Y^ + V) > C • (U + Y + Y^ + V). 
Now Z := F + Y + Y~^ + V is a symmetric nxn matrix which has no negative 
entries and is also positive-semidehnite, as follows from 


y^Zy = [y'^, y^]X[y'^, y^ > 0 for all y eRY 


Further, ee"'^ • Z = E • X, which shows 


=mm^C •{U-2Y+ V) : X = 
> min [C • Z : Z e V n M, ee^ 

Obviously the matrix C of m with a 
have Og®®* ^ —1.1429 whereas Og^ = a* 


U 

Y 

• Z = 


Y 

V 

1 } 


ePnA/', EtX 


= a 


cop 

C 



(14) 

= 0 has no positive entries and we 
= —1 is exact. □ 
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Figure 1: Distribution of for 1000 randomly generated n x n ma¬ 

trices. 



6 Empirical findings 

To compare the quality of the two SDP relaxation bounds we generated 1000 
random symmetric n x n matrices for u = 10 and n = 20 respectively. The 
SDP problems were solved using Jos Sturm’s SeDuMi jH]. To obtain upper 
bounds we applied two optimization procedures for the StQP a hxed-step 
exponential replicator dynamics with 6 = 0.05, and Wolfe’s reduced gradient 
method. For details and comparison with other local optimization procedures 
for StQPs see jl]. For both iterative methods we used 10 random starting 
vectors respectively and took the overall minimum of both procedures as a 
reference solution We thus have and in case of 

equality of the last two bounds we can conclude a’’®^ = a™’’ = a*. 

According to our simulations for n = 10 in more than 97% of all instances 
the copositive bound coincided with the exact solution, whereas for the Nes¬ 
terov bound this was true only in 27% of all cases. There was not a single 
instance where the two SDP relaxation bounds would have coincided while 
being different from a'’®^. To assess the quantitative difference between 
and a^®®* we show descriptive statistics of the ratio of the two bounds in 
Tableland we have plotted the histogram of the ratio in Figure 1. Note 
that for the 3x3 matrix m with a = 1 we had a ratio 0.75, which is already 
quite extreme compared to the minimum 0.69 of our simulations. 

For 20 X 20 matrices we obtained similar results. Still in 95% of all cases 
the copositive bound was exact, whereas for the Nesterov bound this ratio 

8 








dropped to 12%. There was only one instance for which the copositive and the 
Nesterov bound coincided but were smaller than the reference solution. The 
results of Table [Hand the comparison of Figure la and Figure lb indicate that 
for larger dimensions n the discrepancy between the two bounds is increasing. 

Table 1: Descriptive statistics for the ratio for 1000 randomly 

generated n x n matrices. 


n 

Mean 

Std 

Min 

Median 

10 

0.948 

0.058 

0.690 

0.968 

20 

0.896 

0.077 

0.679 

0.899 


7 Extensions for homogeneous quadratic 
optimization over the £^-ball, 1 < p < 2 

We briefly want to address the more general problem 

a* = min {y^Cy : y e Bp} , (15) 

where 1 < p < 2. In the case of positive-semi definite matrices C obviously 
a* = 0 for all p, therefore we concentrate on matrices C ^V. 

In the SDP Handbook [Zj, Nesterov mentions that for 1 < p < 2, no 
practical SDP bounds for m are in sight. Because the considered balls Bp 
are included in the P-hal\ B2, a cheap bound is always given by the eigenvalue 
bound, which is the £^-solution 

Amin(C') = min [y^Cy : y e B2} . 


Obviously this bound is getting worse for p close to 1. However, we can make 
use of to obtain a better bound for small p: 

Theorem 7.1 A valid SDP-based bound for the problem m is given by 


:= n 


2(P-1) 


a 


cop 

Qc 


Proof. We start by blowing up the £^-ball such that the result contains Bp. 

p — 1 

Applying Holder’s inequality we obtain ||p||i < ||pi|p||e||g = with 


1 P —1 

- = 1, and thus it is evident that C n p Bi. Therefore 


Op > min 


P-1 


y Cy : llyll^ <n p \ >n 


2(P-1) 


a 


cop 

Qc 
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Figure 2: Quality of lower bounds in dependence of p for: 


(a) a typical 10 x 10 matrix 




where the last inequality follows from homogeneity of degree two of the 
quadratic form, and from the validity of the copositive bound. □ 


We want to discuss the qualities of the lower bound ap°^ with two exam¬ 
ples. In Figure 2a we consider a randomly generated 10 x 10 matrix which 
illustrates the typical situation: for p close to one our new bound gives a 
considerable improvement compared to the eigenvalue bound, for p = 1 in 
this case it is actually again the exact solution. Denoting the ||.|| 2 -normalized 
eigenvector corresponding to Amin(C') by Umin, we have ||nmin ||p~^n min G Bp, 
hence feasible for m- Based on that observation we can calculate the simple 

A • (O') 

upper bound as plotted in Figure 2a. 

Il’^min lip 

In general will be smaller than Amin(C') for p close to two. But there 
are actually cases where is always larger than Amin(C'), as in Figure 2b 
for the matrix 


C = 


1-11 
-1 11 

1 1 1 


( 16 ) 


In fact, here a^P = a* is the exact solution. The simple upper bound we 

p—2 

considered is given here by 3~ Amin(C')- Now = —1/3 and Amin(C') = —1 
leads to equality of the upper and the lower bound. 
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